Field of Disclosure
This invention relates generally to data mining, and particularly to processing data using data mining operations within a columnar database management system.
Description of the Related Art
Data mining operations are often employed by software application frontends of data mining systems to process data sourced from various database management systems. In particular, software applications typically retrieve data stored by the database management systems, and thereafter perform various calculations on the data according to selected sets of data mining operations (data mining operations are also referred to herein as data mining algorithms). In one aspect, because software applications (representing an application layer) and database management systems (representing a data layer) are logically and often physically distinct entities, processing of data is performed in storage spaces separate from the storage spaces in which the data is stored. As a consequence, performance of data mining processes can frequently be quite poor.
For example, in order to process a data mining request, a software application frontend may transmit a first request for a first set of data to a database management system. In handling the first request, the database management system may authenticate and parse the first request, which delays the overall processing of the data mining request. Upon receiving the first set of data, the software application frontend may generate an intermediate result and thereafter transmit a second request for an additional set of data to the database management system. In handling the second request, the database management system may again authenticate and parse the request, which further delays processing of the data mining request. Finally, after receiving the additional set of data, the database management system processes the additional set and the intermediate result to generate a final result. However, as outlined, generation of such a final result may be relatively slow because several delays associated with request authentication and parsing have been introduced to the data mining process.